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Background / Context: 

Description of prior research and its intellectual context. 

Single-case research (SCR) methods provide a scientifically rigorous approach for documenting 
experimental control and advancing effective practices in education, behavior analysis and 
psychology (Kennedy, 2005; Kratochwill & Levin, 1992; McReynolds & Keams, 1983; 
Richards, Taylor, Ramasamy & Richards, 1999; Tawney & Gast, 1984; Todman & Dugard, 
2001.) Despite its long history and its potential for providing strong evidence for causal 
inference (Shadish & Rindskopf, 2007), single-case research has not collectively advanced 
evidence-based practice in these areas. Part of the reason can be traced to the fact that no 
acceptable statistical procedures are available for the analysis of single subject designs (SSD) 
and no satisfactory mechanism exists for synthesizing the results from single-case research 
studies. 

The uses of single-case research are now expanding, however, and it is necessary to supplement 
visual analysis with a more fonnal index of effect size (Parker & Brossart, 2003; Parker et ah, 
2005). The movement to conduct meta-analyses with a body of research to document the basic 
patterns and findings in a field of study has begun to take hold in single subject research. The 
current commitment to documenting “evidence-based practices” (Kratochwill & Stoiber, 2002; 
Odom et ah, 2005; Whitehurst, 2003) further emphasizes the need to move beyond statements 
based on results from any one study, toward documentation of effects supported by an integrated 
set of research results. Without a meaningful effect size index (Hedges, 2007) such syntheses of 
research and documentation of effective practices cannot take place (Horner, Swaminathan, 
Sugai, & Smolkowski, 2012). 

Arguably, the most fruitful application of statistical procedures to single-case results may lie 
with developing strategies for measuring effect size. Effect size measures the magnitude of the 
difference in results in standardized units. More than 40 approaches for assessing effect size with 
single-case research have been proposed (Parker & Hagan-Burke, 2007; Kirk, 1996), and more 
recent proposals have become increasingly elegant (Parker, Hagan-Burke & Vannest, 2007). At 
this time, however, each of the available approaches carries important shortcomings (Maggin, 
Swaminathan, Rogers, O’Keefe, & Sugai, 2011). Measuring the size of experimental effects in 
single-case research will become a necessary element for the broad application of single-case 
results. To this end, an appropriate effect size measure has to be developed (Beretvas & Cheung, 
2008; Hedges, 2007; Shadish & Rindskopf, 2007; Shadish & Rindskopf, & Hedges, 2008). 

In addition to the problem of defining an index of effect size, the major concern raised by single- 
case researchers is the applicability of statistical procedures that fail to take into account the 
realities of single-subject designs. Serial dependence refers to the extent to which the patterns in 
repeated observations are functions of time separation between a series of values. While initially 
researchers (Huitema, 1985) expressed skepticism regarding the existence of serial dependence, 
researchers including Huitema and colleagues (Huitema, 1986; Huitema &McKean, 2007; 
McKnight, Mckean, & Huitema, 2000) have come to realize that serial dependence is an 
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importance issue that must be taken into account in the statistical analysis of single-subject 
design and have provided procedures that take into account serial dependencies in the data. 
Shadish et al. (2008) have emphasized this fact in their discussion of the methodological issues 
surrounding the analysis of single-subject designs. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

One of the vexing problems in the analysis of SSD is in the assessment of the effect of 
intervention. Serial dependence notwithstanding, the linear model approach that has been 
advanced involves, in general, the fitting of regression lines (or curves) to the set of observations 
within each phase of the design and comparing the parameters of these lines (or curves). In the 
simplest case of an AB design, this involves fitting a regression lines to the observations in 
phases A and B and comparing the slopes and intercepts. In the event the slope is zero in the two 
phases, the comparison reduces to the comparison of the means or levels. If there is a trend but 
the slope is the same in the two phases, then the adjusted means (levels) are compared. A 
problem arises if the levels and the trends in the two phases are different. In this case the 
intercepts and slopes are compared across the phases. While statistically such comparisons are 
not problematic, the interpretations of treatment effects that arise from these comparisons are not 
intuitive. A “combination” of the slope and intercept parameters that yields a single measure of 
treatment effect could be more meaningful. Such a measure of treatment effect yields, in turn, an 
effect size that can be readily computed and interpreted. 

To this end, the study deals with: 

1 . Development of a measure of the effect of intervention which leads to an effect size 
measure 

2. Development of statistical procedures for assessing intervention effects that take into 
account the serial dependence in single-subject designs; 

The proposed statistical procedures are based on linear models, take into account the serial 
dependency in the observations, and are applicable when there is a mean shift as well as trend. In 
addition, an effect size measure that lends itself to meta-analysis is proposed. 

Setting: 

Description of the research location, s 

(May not be applicable for Methods submissions) 

Not Applicable 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 
(May not be applicable for Methods submissions) 

Not Applicable 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and 
duration. 

(May not be applicable for Methods submissions) 

Not Applicable 
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Significance / Novelty of study: 

Description of what is missing in previous work and the contribution the study makes. 

As described in the previous section, this study provides a measure of treatment effect in single- 
subject designs, and consequently an effect size measure which could be used effectively in 
meta-analysis of single subject designs (Maggin, Swaminathan, Rogers, O’Keefe, & Sugai, 
1991). In addition, analytical procedures, classical as well as Bayesian, that take into account 
serial dependency in the observations for testing hypotheses regarding the effect of intervention 
is provided. Furthermore, when multiple subjects are available, a D type effect size estimator is 
obtained employing the within and between subject variation. 

Statistical, Measurement, or Econometric Model: 

Description of the proposed new methods or novel applications of existing methods. 

In the simplest case of an AB design, the linear models fitted to Phases A and B are: 

y? = Pl+Pft + e t (t = l,2....,n A ) 

yf = Pi + Pi t + e, (t = n A + 1, n a + 2 ,....,n A + n B ) 

If we set J3f = J3f = 0 , we obtain a model where there is only a level change between the phases; 
on setting J3f - fif = /?, we obtain the model where there is a shift in the levels adjusted for the 

trend. Following the logic provided in Shadish, Cook, & Campbell (2002) in regards to quasi 
experimental time series designs, we conclude that if the trend lines in the two phases coincide, 
the intervention is not effective. This the effect of the intervention may be defined as the 
difference between the pints on the trend line in Phase B and the points of the Phase A trend line 
that is projected into Phase B. At point any point ha +t in phase B, the difference between the 
trend line in Phase B and the projected trend line from Phase A is 

= y Bm) - y B ,,(o = (Pi - Pi) + (Pi - Pi) * (n A + 0 

This difference is calculated at all points in Phase B and averaged to yield the treatment effect 
defined as 

a „ = ( K - PS ) + (A* - K r — Vf - 1} • 

The estimate of the treatment effect measure defined above is obtained by substituting the 
estimates of the regression coefficients. It follows that if A = 0 we can conclude that the 
intervention is not effective. To test the hypothesis A=0 (against the alternative, A ^ 0\, the ratio 

D / SE(D) is fonned and has an approximate /-distribution with degrees of freedom ( N - 5), in 
the presence of first order autoregressive errors. 

In the estimation of the regression coefficients and consequently D, and in particular, the 
standard error of D, the serial dependency in the observations must be taken into account. 
Maximum Likelihood and the Cochran-Orcutt procedures are classical approaches that can be 
employed to estimate the autocorrelation parameter, the regression coefficients, the error 
variance, and finally, D. Asymptotic as well as conditional standard errors are computed for 
testing the hypothesis A =0 and for constructing confidence intervals around A . 
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The parameter A is clearly an effect size measure. A standardized effect size measure is obtained 
by dividing by the standard deviation obtained by pooling the error variances in the two phases. 
An approximate distribution of the standardized effect size measure is obtained in this study. 

An alternative and a more fruitful approach is to employ a Bayesian procedure to obtain the 
posterior distribution of the regression coefficients and A ab , the treatment effect parameter. 

The advantage of this approach is that the posterior distribution contains all the infonnation 
needed for drawing inferences. By computing the 'A a and (1 -Via) percentile points, {\-a )% 
credibility intervals can be constructed without making any distributional assumptions. Markov- 
Chain Monte Carlo (MCMC) procedure as implemented in WINBUGS is employed for 
obtaining the Bayesian estimates. 

In the event several subjects are included in the study as in multiple baseline designs, a Bayesian 
procedure that in principle equivalent to hierarchical linear modeling approach, is proposed in 
the study. This approach takes into account the intra and inter subject variation to provide a 
measure of variability that yields a D-type effect size estimator. This approach is illustrated using 
the MCMC procedure as implemented in WINBUGS. 


Usefulness / Applicability of Method: 

Demonstration of the usefulness of the proposed methods using hypothetical or real data. 

The methods described above are applied for the analysis of several standard sets of data so that 
the different approaches proposed by the symposium presenters for analyzing single subject 
designs can be compared. Classical and Bayesian procedures are compared. 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

The statistical procedure developed here that includes a definition of treatment effect and 
consequently an effect size provides a viable method for the analysis of single-subject designs. 
Then advantage of the procedure outline here is that a single measure of treatment effect and 
consequently a single effect size measure is available when there is both a level change and trend 
change in the phases. Furthennore, the serial dependency in the observations is taken into 
account in the analysis of the data. 

Single-subject designs are very short time series and this presents a problem in estimating the 
parameters in the autoregressive process assumed for the errors. The Bayesian approach 
described has the potential for overcoming this limitation. 
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